college entrance exam
Evaluating Large Language Models: A Comprehensive Survey
Guo, Zishan, Jin, Renren, Liu, Chuang, Huang, Yufei, Shi, Dan, Supryadi, null, Yu, Linhao, Liu, Yan, Li, Jiaxuan, Xiong, Bojian, Xiong, Deyi
Large language models (LLMs) have demonstrated remarkable capabilities across a broad spectrum of tasks. They have attracted significant attention and been deployed in numerous downstream applications. Nevertheless, akin to a double-edged sword, LLMs also present potential risks. They could suffer from private data leaks or yield inappropriate, harmful, or misleading content. Additionally, the rapid progress of LLMs raises concerns about the potential emergence of superintelligent systems without adequate safeguards. To effectively capitalize on LLM capacities as well as ensure their safe and beneficial development, it is critical to conduct a rigorous and comprehensive evaluation of LLMs. This survey endeavors to offer a panoramic perspective on the evaluation of LLMs. We categorize the evaluation of LLMs into three major groups: knowledge and capability evaluation, alignment evaluation and safety evaluation. In addition to the comprehensive review on the evaluation methodologies and benchmarks on these three aspects, we collate a compendium of evaluations pertaining to LLMs' performance in specialized domains, and discuss the construction of comprehensive evaluation platforms that cover LLM evaluations on capabilities, alignment, safety, and applicability. We hope that this comprehensive overview will stimulate further research interests in the evaluation of LLMs, with the ultimate goal of making evaluation serve as a cornerstone in guiding the responsible development of LLMs. We envision that this will channel their evolution into a direction that maximizes societal benefit while minimizing potential risks. A curated list of related papers has been publicly available at https://github.com/tjunlp-lab/Awesome-LLMs-Evaluation-Papers.
SoftBank CEO: Japan should make AI a mandatory subject for college entrance exams
Japanese students "don't study if they are not asked … let's put it as mandatory, then Japanese students will catch up," Son told a government conference aimed at fostering innovation. The comments came as Son pointed to the widening gap in GDP and AI-related patents filed in Japan compared to China and the United States. "Japan has lost the past, but may [also] be losing the future," he said. Son said Japan should focus on two areas -- autonomous driving and DNA-centered medicine -- to help combat the pressures of its rapidly aging society, which is seeing a rise in traffic accidents involving elderly drivers and health care costs. "Even today's technology of autonomous driving is better than senior citizens driving on the street," Son said.
Super-intelligent machines: AI may soon pass Japan's toughest test
A robot developed by the National Institute of Informatics is now smart enough to be accepted into most Japanese universities - but not the notoriously selective University of Tokyo. This artificial intelligence is called the Todai Robot Project, and aims to pass the entrance exam for the University of Tokyo in 2021. For the first time in its development, the AI program achieved an above-average score on a college entrance exam, which covered maths, physics, and english among other subjects. University of Tokyo, also called'Todai,' requires prospective students to take the general admissions test, the National Center Test for University Admissions, along with its own infamously difficult test The University of Tokyo, also referred to as'Todai,' is notorious for its extremely difficult entrance exam. Prospective students must take a general college entrance exam, the National Center Test for Admissions, along with the Todai test.